Information processing

ABSTRACT

Disclosed are methods and apparatus for displaying information relating to a portion of a multimedia presentation. A progress bar is provided and displayed. A length of the progress bar is representative of a duration of the multimedia presentation. A user selects a first and a second point in the multimedia presentation, thereby defining a portion of the presentation. Depending on the selection, one or more communications (e.g., comments, online postings, etc.) may be identified. Each of these communications may have been composed in relation to at least a point of the multimedia presentation that is within the defined portion. Information depending on those identified communications may then be displayed to the user.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is related to U.S. patent applications (MotorolaDocket Numbers CS39210, CS39217, and CS39294), filed on an even dateherewith.

FIELD OF THE INVENTION

The present invention is related generally to displaying a plurality ofpresentation elements.

BACKGROUND OF THE INVENTION

The consumption of media presentations by users (i.e., media consumers)is common. Here, “media presentation” or “multimedia presentation”refers to any digital content, including but not limited to video,audio, and interactive files. Also, “consumption” refers to any type ofhuman interaction with a media presentation, including but not limitedto watching, listening to, and interacting with the presentation.

Users can provide commentary upon points or periods of interest in amultimedia presentation. For example, a user may comment on an eventoccurring in the media presentation that is meaningful in the context ofthe multimedia presentation. Typically, users communicate these commentsto other users. This communication, or sharing, of a user's comments maybe achieved via, for example, online social media services (e.g., socialnetworking websites), weblogs (i.e., “blogs”), and online forums.

BRIEF SUMMARY

The above considerations, and others, are addressed by the presentinvention, which can be understood by referring to the specification,drawings, and claims. According to aspects of the present invention,information relating to a portion of a multimedia presentation isdisplayed (e.g., to a user on an end-user device such as a touch-screencomputer).

A progress bar is displayed. A length of the progress bar isrepresentative of duration (i.e., a length or domain) of a multimediapresentation. The progress bar may comprise a marker which moves alongthe length of the progress bar as the multimedia presentation isconsumed by the user. A position of the marker along the length of theprogress bar indicates a particular point in the multimediapresentation. A user may select a first and a second point in themultimedia presentation, thereby defining a portion of the multimediapresentation (between the selected points). This selection of pointswithin the multimedia presentation may be performed by selecting pointson the progress bar, for example, by using a “pinching” action on atouch-screen display of an end-user device. In other words, the user mayselect two time-steps by touching a touch-screen display with two of hisdigits on the same hand (e.g., thumb and forefinger). These selectionsmay be changed as desired by the user moving his digits in relation toone another (e.g., in a pinching action).

Depending on the selected time-steps, one or more communications maythen be identified. Each of these communications may originate from arespective originator (e.g., a consumer who has previously consumed themultimedia presentation). Also, each of these communications may havebeen composed in relation to at least a point of the multimediapresentation that is within the portion of the multimedia presentationdefined by the first and second points, i.e., each identifiedcommunication may have been made in relation to a point in themultimedia presentation that is between the selected points.

Information depending on those identified communications may then bedisplayed to the user. This information may be, for example, the one ormore identified communications (e.g., comments, online postings, etc.)made in relation to the defined portion of the multimedia presentation,a summary of one or more of those communications, or a summary of thedefined portion of the multimedia presentation (e.g., based on one ormore of the communications).

Thus, a summary of the events in the selected section of the multimediapresentation may be generated.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

While the appended claims set forth the features of the presentinvention with particularity, the invention, together with its objectsand advantages, may be best understood from the following detaileddescription taken in conjunction with the accompanying drawings ofwhich:

FIG. 1 is a schematic illustration (not to scale) showing a TV program;

FIG. 2 is a schematic illustration (not to scale) of an example network;

FIG. 3 is a schematic illustration (not to scale) of a video server;

FIG. 4 is a schematic illustration (not to scale) of a multimediaserver;

FIG. 5 is a schematic illustration (not to scale) of a tablet computer;

FIG. 6 is a process flow chart showing certain steps of a methodperformed by the entities in the network;

FIG. 7 is a schematic illustration (not to scale) of a first userinterface;

FIG. 8 is a process flow chart showing certain steps of a method ofusing comment and time-stamps information;

FIG. 9 is a schematic illustration (not to scale) of a second userinterface;

FIG. 10 is a process flow chart showing certain steps of a method bywhich a first timestamp may be edited;

FIG. 11 is a further schematic illustration (not to scale) of the seconduser interface;

FIG. 12 is a process flow chart showing certain steps of an examplemethod by which a comment may be associated with multiple firsttimestamps;

FIG. 13 is a further schematic illustration (not to scale) of the seconduser interface;

FIG. 14 is a process flow chart showing certain steps of an examplemethod of specifying a portion of the TV program;

FIG. 15 is a further schematic illustration (not to scale) of the seconduser interface; and

FIG. 16 is a schematic illustration (not to scale) of a further examplenetwork.

DETAILED DESCRIPTION

Turning to the drawings, wherein like reference numerals refer to likeelements, the invention is illustrated as being implemented in asuitable environment. The following description is based on embodimentsof the invention and should not be taken as limiting the invention withregard to alternative embodiments that are not explicitly describedherein.

Apparatus for implementing any of the below described arrangements, andfor performing any of the below described method steps, may be providedby configuring or adapting any suitable apparatus, for example one ormore computers or other processing apparatus or processors, or providingadditional modules. The apparatus may comprise a computer, a network ofcomputers, or one or more processors, for implementing instructions andusing data, including instructions and data in the form of a computerprogram or plurality of computer programs stored in or on amachine-readable storage medium such as computer memory, a computerdisk, ROM, PROM, etc., or any combination of these or other storagemedia.

It should be noted that certain of the process steps depicted in thebelow described process flowcharts may be omitted or such process stepsmay be performed in differing order to that presented below and shown inthose process flowcharts. Furthermore, although all the process stepshave, for convenience and ease of understanding, been depicted asdiscrete temporally-sequential steps, nevertheless some of the processsteps may in fact be performed simultaneously or at least overlapping tosome extent temporally.

Referring now to the Figures, FIG. 1 is a schematic illustration (not toscale) showing a TV program 2. The TV program 2 is used herein toillustrate embodiments of the invention. It will be appreciated that inother embodiments a different type of media presentation or multimediacontent (e.g., a radio broadcast, an ebook) may be processed. The TVprogram 2 may be consumed by the user using any appropriate clientdevice (e.g., a laptop computer, a tablet computer, a television). TheTV program 2 runs from its start time 4 at time t=0 to its end time 6 attime t=T. In embodiments in which the media presentation is, e.g., anebook, the media presentation may run from ‘page l’ to ‘page N’ (where Nis the number of pages in the ebook). Thus, as opposed to being atime-based medium, a media presentation may be non-time-based. The mediapresentation may also comprise a multi-device presentation, i.e., amultimedia presentation to be shown on more than one different device.For example, the media presentation may comprise a slide presentation tobe shown by one device and a video presentation to be shown by anotherdevice.

A portion of the TV program 2 (hereinafter referred to as the “portion”and shown in FIG. 1 as a shaded region of the TV program 2 and indicatedby the reference numeral 8) has start and end demarcation points(indicated in FIG. 1 by the reference numerals 10 and 12 respectively).The start demarcation point 10 of the portion 8 occurs in the TV program2 at time t=t1. The end demarcation point 12 of the portion 8 occurs inthe TV program 2 at time t=t2. The start demarcation point 10 may be anytime point in the TV program 2 between t=0 and t=T. The end demarcationpoint 12 may occur at any time point in the TV program that is equal toor later than the start demarcation point 10.

FIG. 2 is a schematic illustration (not to scale) of an example network14. A process, as performed by the entities of the network 14, isdescribed in more detail below with reference to FIG. 6.

The representative network 14 comprises a service operator 16, a videoserver 18, a multimedia server 19, a set-top box 20, a television 22, auser 24, the Internet 26, a database 28, and a tablet computer 30.

The service operator 16 comprises apparatus that provides a televisionfeed corresponding to the TV program 2. The service operator 16 iscoupled to the video server 18 (e.g., either by a wireless or wiredconnection) such that, in operation, the service operator 16 providesthe television feed to the video server 18.

The video server 18 (e.g., a cable head-end) is a facility forreceiving, processing, and re-distributing television signals (e.g., thetelevision feed). In addition to being coupled to the service operator16, the video server 18 is coupled (e.g., either by a wireless or wiredconnection) to the multimedia server 19 and to the set-top box 20.

In operation, the video server 18 inter alia receives the televisionfeed from the service operator 16. The video server 18 then processesthe received television feed and distributes the processed feed (e.g.,encoded in an appropriate multimedia container) to the multimedia server19 and to the set-top box 20. The video server 18 and its functionalityare described in more detail below with reference to FIGS. 3 and 6.

In addition to being coupled to the video server 18, the multimediaserver 19 is coupled to the Internet 26, to the database 28, and to thetablet computer 30. The multimedia server 19 and its functionality aredescribed in more detail below with reference to FIGS. 4 and 6.

In addition to being coupled to the video server 18, the set-top box 20is also coupled to the TV 22. The set-top box 20 is a conventionaldevice that, in operation, processes a multimedia container receivedfrom the video server 18 to provide content for display on the TV 22 (tothe user 24).

The TV 22 is a conventional television on which, in operation, contentfrom the set-top box 20 is displayed to the user 24.

The user 24 is a user of the TV 22, the database 28, and the tabletcomputer 30. For example, the user 24 may watch TV programs on the TV22, store personal information in electronic form in the database 28,and browse the Internet 26 using the tablet computer 30. Furthermore,the user 24 is a subscriber to a service that allows the user 24 toupload comments and relevant information (e.g., onto the multimediaserver 19) relating to media presentations that he has consumed or iscurrently consuming. For example, whilst the user 24 watches the TVprogram 2, the user 24 is able to upload (in electronic form) commentsand content items relating to the TV program 2, e.g., relating to eventsoccurring in the TV program 2, to the multimedia server 19 via theuser's tablet computer 30 (as described in more detail below withreference to FIG. 6). The information uploaded to the multimedia server19 that relates to a particular media presentation may be provided (bythe video server 18) for consumption by other consumers of the mediapresentation, as described in more detail below with reference to FIG.7. This service further allows the user 24 to access informationuploaded by other parties (e.g., as described below with reference toFIG. 7).

The database 28 is storage for personal electronic information of theuser 24. In other words, the database 28 is used by the user 24 to storethe user's personal electronic content. The personal content stored bythe user may include, but is not limited to, photographs, home-movies,and text documents. The database 28 may reside in a device used, owned,or operated by the user 24, for example a computer of the user 24, e.g.,a laptop computer or the tablet computer 30.

The tablet computer 30 is a conventional tablet computer. In addition tobeing coupled to the multimedia server 19, the tablet computer 30 hasaccess to the Internet 26 (e.g., via Wi-Fi Internet access). In otherembodiments, a different type of device may replace the tablet computer30, e.g., a different type of computer such as a laptop computer or a“smartphone.” The tablet computer 30 is configured to allow the user 24to input a comment relating to a media presentation (e.g., the TVprogram 2) that the user 24 has consumed or is consuming. For example,the user 24 may type such a comment on to the tablet computer 30. Thetablet computer 30 may then upload the user's comment and content itemsonto the multimedia server 19, e.g., via a wired or wireless link, orvia the Internet 26. The tablet computer is described in more detailbelow with reference to FIG. 5.

FIG. 3 is a schematic illustration (not to scale) of the video server18.

The video server 18 comprises a Transport Stream (TS) encoder 32, atransmitter 34, an Electronic Program Guide (EPG) service module 36, aclosed-caption service module 38, and a media analysis module 40.

The TS encoder 32 is for encoding information into an appropriateMultimedia Container for transmission to the set-top box 20. The TSencoder 32 is coupled to the service provider 16 and to the transmitter34 such that the TV feed may be received by it from the service provider16, encoded, and delivered to the transmitter 34.

The transmitter 34 is coupled to the set-top box 20 such thatinformation (e.g., the TV feed) encoded by the TS encoder 32 istransmitted to the set-top box 20.

The EPG service module 36 is a provider of broadcast programming orscheduling information for current and upcoming TV programs. The EPGservice module 36 is coupled to the media analysis module 40 such thatinformation may be sent from the EPG service module 36 to the mediaanalysis module 40.

The closed-caption service module 38 is a provider of closed captioninformation (i.e., sub-titles) for current and upcoming TV programs. Theclosed captions may be in any appropriate language. The closed-captionservice module 38 is coupled to the media analysis module 40 such thatinformation may be sent from the closed-caption service module 38 to themedia analysis module 40.

In addition to being coupled to the EPG service module 36 and to theclosed-caption service module 38, the media analysis module 40 iscoupled to the service operator 16 such that, in operation, it receivesthe TV feed provided by the service module 16. The media analysis module40 is for analyzing and processing the received television feed,programming information, and closed caption information, as described inmore detail below with reference to FIG. 6. Also, the media analysismodule 40 is coupled to the multimedia server 19 such that output fromthe media analysis module 40 may be sent to the multimedia server 19.

FIG. 4 is a schematic illustration (not to scale) of the multimediaserver 19.

The multimedia server 19 comprises a content recommendation module 42, acomment service module 44, a comment analysis module 46, and a furtherdatabase 48.

The media analysis module 40 (of the video server 18) is coupled to thecontent recommendation module 42 such that output from the mediaanalysis module 40 may be sent to the content recommendation module 42.

In addition to being coupled to the media analysis module 40, thecontent recommendation module 42 is coupled to the comment analysismodule 46. This is so that the content recommendation module 42 mayreceive and process the output from the media analysis module 40 andoutput from the comment analysis module 46. Also, the contentrecommendation module 42 is coupled (e.g., by a wired or wired-lessconnection) to the Internet 26 and to the database 28. This is so thatthe content recommendation module 42 may retrieve content over theInternet 26 (e.g., from a web-server not shown in the Figures) and mayretrieve content (e.g., the user's personal pictures or movies) from thedatabase 28. Also, the content recommendation module 42 is coupled tothe comment service module 44. This is so that output from the contentrecommendation module 42 may be sent to the comment service module 44.The operation of the content recommendation module 42 is described inmore detail below with reference to FIG. 6.

In addition to being coupled to the content recommendation module 42,the comment service module 44 is coupled (e.g., by a wired or wired-lessconnection) to the tablet computer 30. This is so that information maybe sent from the tablet computer 30 to the comment service module 44 andvice versa. In other embodiments, the comment service module 44 may alsobe coupled to the TV 22 such that information may be sent from the TV 22to the comment service module 44 and vice versa. Also, the commentservice module 44 is coupled to the comment analysis module 46 and tothe further database 48. This is so that information may be sent fromthe comment service module 44 to each of the comment analysis module 46and the further database 48, as described in more detail below withreference to FIG. 6.

The comment analysis module 46 is for analyzing and processing an outputof the comment service module 44, as described in more detail below withreference to FIG. 6. In operation, an output of the comment analysismodule 46 is sent from the comment analysis module 46 to the commentrecommendation module 42.

The further database 48 is for storing data received by it from thecomment service module 44.

FIG. 5 is a schematic illustration (not to scale) of the tablet computer30.

The tablet computer 30 comprises a processor 80 and a display 82(operatively connected to the processor 80).

The processor 80 is connected (e.g., via a wireless link) to theInternet 26 and to the multimedia server 19 so that the processor 80 mayreceive information over the Internet 26 and from the multimedia server19. Also, the processor 80 is configured to send (i.e., transmit)information over the Internet 26 and to the multimedia server 19. Thus,the processor 80 acts as a transmitting module and a receiving modulefor the tablet computer 30.

The display 82 is a touch-screen display. The user 24 may inputinformation to the tablet computer 30 using the display 82.

The processor 80 and the display 82 are coupled such that the processor80 may send information to the display 82 for displaying to the user 24.Also, the processor 80 may receive information input to the tabletcomputer 30 by the user 24 using the display 82.

The processor 80 is arranged to process received information inaccordance with the below described methods.

FIG. 6 is a process flow chart showing certain steps of a methodperformed by the entities in the network 14.

At step s2, the TV feed corresponding to the TV program 2 is sent fromthe service operator 16 to the TS encoder 32 and to the media analysismodule 40 of the video server 18.

After step s2 the method proceeds to both step s4 and step s24. Step s24is described in more detail below after a description of steps s4through s22.

At step s4, the TS encoder 32 encodes the television feed for the TVprogram 2 that it received from the service operator 16. The encodedtelevision feed may be inserted into an appropriate MultimediaContainer. Examples of a Multimedia Container include the MPEG TransportStream, Flash Video, and QuickTime. The MPEG Transport Stream maycomprise, for example, audio elementary streams, video elementarystreams, closed-caption or subtitle elementary streams, and a ProgramAddress table. The TV feed, as it may comprise text, audio, video, andother type of information (such as graphical composition and animationinformation), may be separately carried by other types of elementalstreams in the same transport stream, such as the video, audio, text,and private user data elemental streams.

At step s6, the encoded Multimedia Container is sent from the TS encoder32, via the transmitter 34, to the set-top box 20.

At step s8, the set-top box 20 decodes the received MultimediaContainer.

At step s10, the content (from the decoded Multimedia Container), i.e.,the TV feed, is displayed to the user 24 on the TV 22.

At step s12, as the user 24 watches the TV program 2, the user 24 types(or inputs in some other way) a comment relating to the TV program 2(e.g., to events occurring in the TV program 2) on to the tabletcomputer 30 (using the display 82).

At step s14, the user's comment relating to the TV program 2 istransmitted from the tablet computer 30 (e.g., via the processor 80) tothe comment service module 44 of the multimedia server 19.

The comment may be sent from the tablet computer 30 as the user 24 isinputting the comment, i.e., a partial comment may be transmitted. Thismay be done such that the steps s16 through s38 described below areperformed whilst the user 24 is still composing the comment.Alternatively the comment may be transmitted from the tablet computer 30after the user 24 has finished composing it.

Furthermore, at step s14, a first time-stamp relating to the comment istransmitted from the tablet computer 30 to the comment service module 44of the multimedia server 19. This first time-stamp is indicative of apoint in time in the TV program 2 (i.e., a time between the start time4, t=0, and the end time 6, t=T, of the TV program 2) corresponding towhich the user 24 makes the comment. For example, the first time-stampmay indicate the time in the TV program 2 that a certain event occurs,the certain event being an event with respect to which the user 24 makesthe comment.

Furthermore, at step s14, a second time-stamp relating to the comment istransmitted from the tablet computer 30 to the comment service module 44of the multimedia server 19. This second time-stamp is indicative of adate and time of day at which the user 24 makes the comment.

At step s16, the user's comment and the associated first and secondtime-stamps are stored by the comment service module 44 at the furtherdatabase 48.

At step s18, the user's comment is sent from the comment service module44 to the comment analysis module 46.

At step s20, the comment analysis module 46 analyses the received user'scomment. Any appropriate process for analyzing the user's comment may beimplemented. For example, conventional keyword detection processes andparsing processes may be used to analyze a text form of the user'scomment.

At step s22, the results of the analysis of the user's comment performedby the comment analysis module 46 (e.g., the extracted key-words or theparsed comment) is sent from the comment analysis module 46 to thecontent recommendation module 42.

After step s22, the method proceeds to step s32, which is describedbelow after the description of steps s24 through s30.

At step s24, the EPG service module 36 sends metadata corresponding tothe TV program 2 (e.g., the start time 4 and end time 6 of the TVprogram 2, the type of the TV program 2, genre of the TV program 2, castand crew names of the TV program 2, the parental advisory rating of theTV program 2, etc.) to the media analysis module 40.

At step s26, the closed-caption service module 38 sends closed captioninformation for the TV program 2, in the appropriate languages, to themedia analysis module 40.

At step s28, using the metadata received from the EPG service module 36,the closed caption information received from the closed-caption servicemodule 38, and the TV feed received from the service operator 16, themedia analysis module 40 analyses the television feed to detect contentevents, hereinafter referred to as “events,” within the TV program 2.The terminology “event” is used herein to refer to a point or period ofinterest in a multimedia presentation (e.g., the TV program 2). Thepoint or period of interest in the multimedia presentation is meaningfulin the context of the multimedia presentation. Examples of eventsinclude an action sequence in a movie, a home-run in a televisedbaseball game, a goal in a televised soccer match, and a commercialbreak in a TV program. The analysis performed by the media analysismodule 40 is any appropriate type of analysis such that events occurringwithin the TV program 2 may be detected. For example, a conventionalprocess of analyzing audio, video, or closed caption (sub-title) text todetect content events may be used. For example, a period of increasedsound levels in a televised soccer match tends to be indicative of anevent (e.g., a goal). Also, for example, an instance of a completelyblack screen in a movie tends to be indicative of an event (e.g., a cutfrom one scene to the next).

At step s30, information relating to one or more of the detected events(e.g., the type of event, closed caption data corresponding to an event,the time the event occurred with respect to the TV program, the durationof the event, etc.) is sent from the media analysis module 40 to thecontent recommendation module 42.

At step s32, using the comment analysis received from the commentanalysis module 46 and the event information received from the mediaanalysis module 40, the content recommendation module 42 identifies oneor more subjects, or topics, related to the user's comment or related tothe event occurring in the TV program 2 that the user 24 is commentingon. This may be performed, for example, by comparing the analysis of theuser's comment with the information relating to that event. For example,keywords extracted from the user's comment may be compared to the closedcaptions that relate to that event or to a text description of thatevent.

The identification of the one or more topics may further depend on userbehaviour (e.g., the user's web-browsing history, etc.) or properties ofthe user 24 (e.g., a profile of the user 24, the user's likes, dislikes,hobbies, interests, etc.). The identification of the one or more topicsmay instead, or also, depend upon properties of the intended recipientsof the comment (i.e., the party or parties that the user 24 intends,i.e., has indicated, the comment to be viewed by). The identification ofthe one or more topics may instead, or also, depend upon previouscomments made by the user or by a different party (e.g., made inrelation to the multimedia presentation).

At step s34, the content recommendation module 42 identifies contentthat relates to the one or more subjects or topics identified at steps32. This may be performed by the content recommendation module 42searching on the Internet 26 (e.g., using a Web search engine) forinformation related to those subjects. Also, the content recommendationmodule 42 may search the database 28 for information (e.g., pictures ordocuments) related to those subjects. The content recommendation module42 selects some or all of the identified content items. For example, thecontent recommendation module 42 may download some or all of theidentified content items. Also, the content recommendation module 42 maycompile a list of links or references (e.g., hyperlinks) to some or allof the identified content items.

The selection of some or all of the identified content items may dependon one or more other factors, e.g., user behaviour (e.g., the user'sweb-browsing history, etc.), properties of the user 24 (e.g., a profileof the user 24, the user's likes, dislikes, hobbies, interests, etc.),properties of the intended recipients of the comment (i.e., the party orparties that the user 24 intends, i.e., has indicated, the comment to beviewed by), or previous comments made by the user or by a differentparty (e.g., made in relation to the multimedia presentation).

The set of all the downloaded or selected content items and the list ofreferences is hereinafter referred to as the “recommended content.” Theterminology “recommended content” refers to content that is identifiedand selected by the content recommendation module 42 and will berecommended to the user 24 as being relevant to, e.g., the comment orthe event in relation to which the comment is made. The recommendedcontent may include, but is not limited to, links to websites, newsarticles, and downloaded documents or pictures (e.g., maps downloadedfrom a web-server via the Internet 26 or personal pictures retrievedfrom the database 28).

The recommended content may also comprise, for one or more of therecommended content items (i.e., the downloaded content items and therecommended links) a recommended time in the TV program 2. A recommendedtime for a content item may be a time in the TV program that an eventoccurs to which the content item is related. A recommended time for acontent item may be determined by the content recommendation module 42(or by a different module) by, for example, analyzing the commentanalysis (i.e., the output of the comment analysis module 46) and theevent information (i.e., the output from the media analysis module 40).Also for example, a recommended time for a content item may bedetermined by analyzing comments of other users (e.g., that may bestored in the further database 48).

At step s36, the recommended content is sent from the contentrecommendation module 42 to the tablet computer 30 (via the commentservice module 44). In other embodiments, the recommended content issent from the content recommendation module 42 to the TV 22 for displayto the user 24.

At step s38, the recommended content is displayed to the user 24 on thetablet computer 30 (on the display 82). In other embodiments, therecommended content is displayed on a different device, e.g., the TV 22.The recommended content may be displayed to the user 24 whilst the user24 is still composing the comment or after the user 24 has initiallyfinished composing the comment.

FIG. 7 is a schematic illustration (not to scale) of an example of auser interface (UI), hereinafter referred to as the “first UI” andindicated in FIG. 7 by the reference numeral 50. The first UI 50 isdisplayed on the display 82 of the tablet computer 30. The first UI 50comprises a progress bar 54 that conveys to the user 24 how much of theTV program 2 has been played and how much of the TV program 2 remains tobe played. A shaded portion 56 of the progress bar 54 increases inlength, along the length of the progress bar 54, from one end of theprogress bar 54 (which corresponds to the start time 4, t=0, of the TVprogram 2) to the other end of the progress bar 54 (which corresponds tothe end time 6, t=T, of the TV program 2). The first UI 50 furthercomprises a comment box 58 in which the user 24 types his comment 59.The first UI 50 further comprises a recommendations box 60 in which therecommended content 62 relating to the comment 59 in the comment box 58(received at the tablet computer 30 from the content recommendationmodule 42) is displayed.

In other examples, a number of comments made by the user 24 may bedisplayed in the first UI 50 (e.g., in a number of other comment boxes).Recommended content relating to some or all of these other comments mayalso be displayed in the first UI 50 (e.g., in a number of otherrecommendations boxes).

Thus, content items that relate to a comment that the user 24 is in theprocess of composing may advantageously be displayed to the user 24 (asrecommended content) as that comment is being composed. The user 24 mayselect (in a relatively simple and intuitive way) one or more of therecommended items for inclusion in the comment. The recommended contentitems advantageously tend to relate to both the comment being composedand to the events occurring in the TV program 2 at that time.

Returning now to FIG. 6, at step s40 the user 24 includes some or all ofthe recommended content 62 in his comment 59. The user 24 may decide toinclude some, all, or none of the recommended content 62 into hiscomment 59. For example, the user 24 may, using the first UI 50, selectthe content he wishes to include in the comment 59 from therecommendations box 60 and “drag-and-drop” that content into the commentbox 58. Thus, the user 24 updates the comment to include recommendedcontent (e.g., web-links to websites or personal pictures that arerelevant to the comment 59 or to the event with respect to which thecomment 59 was made). Also for example, in the first UI 50 a recommendedcontent item 62 may be accompanied by a respective “Attach” button,which when selected by the user 24 results in that content item 62 beingincluded in the comment box 58. In other embodiments, the user 24 maychoose to attach one or more of the recommended content items 62 to thecomment 59, e.g., by touching, pointing, or speaking a correspondingindex. A list of recommended content is automatically updated, i.e., therecommended content list is refined, as the user 24 provides moreinformation or edits the comment 59.

At step s42, the updated comment is sent from the tablet computer 30 tothe comment service module 44.

At step s44, the comment stored in the further database 48 at step s16is updated so that it is the same as the updated comment, i.e., suchthat it contains any recommended content included by the user 24 at steps40. This may be achieved in any appropriate way, e.g., by overwritingthe original comment with the updated comment. In other embodiments, theoriginal comment is not stored at step s16, and only once the commenthas been updated to include any recommended content items desired by theuser 24 is the comment stored.

Thus, at step s44 the updated comment (including recommended content)and the first and second time-stamps corresponding to that comment arestored at the multimedia server 19 in the further database 48.

Using the method described above with respect to FIG. 6, the user 24 maycreate one or more comments relating to the TV program 2. These commentsmay include content recommended to the user 24 by the contentrecommendation module 42 and that relates to the user's comment orevents in the TV program 2 upon which the user 24 comments. Each commentis stored with a time in the TV program 2 at which that comment was made(i.e., a time relative to the TV program 2) and an absolute time (e.g.,data and time) at which that comment was made.

Next described is an example way of using the information stored in thefurther database 48 (i.e., the stored comment and time-stamps).

FIG. 8 is a process flow chart showing certain steps in an examplemethod of using the comment and time-stamps stored in the furtherdatabase 48.

At step s46, a further user records (e.g., using a digital videorecording device) the TV program 2. The further user is a subscriber tothe service that allows him to access comments stored on the furtherdatabase 48 (made, e.g., by the user 24) and to upload his own commentsto the further database 48. The TV program 2 is to be watched by thefurther user at a time that is after the TV program 2 has been watchedand commented on by the user 24.

At step s48, sometime after the TV program 2 has been recorded by thefurther user, the further user watches (e.g., on a further TV) therecording of the TV program 2.

At step s50, whilst watching the recording of the TV program 2, thefurther user displays (e.g., on a further tablet computer or othercompanion device, of which he is a user) the information stored in thefurther database 48. This information may be displayed as describedbelow with reference to FIG. 9.

FIG. 9 is a schematic illustration (not to scale) of an example of auser interface, hereinafter referred to as the “second UI” and indicatedin FIG. 9 by the reference numeral 60.

Similar to the first UI 50 described above with reference to FIG. 7, thesecond UI 60 comprises the progress bar 54 that conveys to the furtheruser how much of the TV program 2 has been played and how much remainsto be played (this is indicated by means of the shaded portion 56).

The second UI 60 further comprises a comment box 58 which displays theuser's updated comment 64 (i.e., the user's comment including anyrecommended content he wishes to include). The comment box 58 alsodisplays the first time-stamp 66 (i.e., the time-stamp indicative of thetime relative to the TV program 2 at which the user 24 makes thecomment). The comment box 58 comprises a marker 68. The position of thismarker 68 with respect to the progress bar 54 corresponds to theposition of the first time-stamp 66 in the TV program 2. In other words,if the first time-stamp 66 indicates that the updated comment 64corresponds to a time t=ti in the TV program 2, then the marker 68 ofthe comment box 58 that contains the updated comment 64 is aligned withthe time t=ti of the progress bar 54. Also, the comment box 58 displaysthe second time-stamp 70 (i.e., the time-stamp indicative of the dateand time of day at which the user 24 made the updated comment 64).

Thus, as the further user watches the TV program 2, he is able to viewthe comments the user 24 made on the TV program 2 (including any contentthe user 24 included in the comments). The further user is also able toview the time with respect to the TV program 2 each comment was made bythe user 24. The further user is also able to view the respectiveabsolute time (i.e., date and time of day) at which each comment wasmade by the user 24. Thus, the user 24 is able to communicate hisopinions on the TV program 2 (and any related content items) relativelyeasily to the further user.

The second UI 60 advantageously provides a relatively simple andintuitive user interface. This interface may be used by a user to viewor interact with comments and other information provided by other usersand relating to a media presentation. The provided comments and relatedinformation may be consumed by a user before, during, or after theconsumption of the related media presentation.

The second UI 60 further provides an interface using which the user 24may edit a comment or a timestamp associated with a comment. The commentto be edited, or the comment associated with the timestamp to be edited,may have been made by any user (i.e., by the user performing the editingor by a different user). Methods of performing such editing aredescribed below with reference to FIGS. 10 through 13.

FIG. 10 is a process flow chart showing certain steps of a method bywhich the user 24 may edit a first timestamp 66 associated with acomment. This may be done by interacting with the second UI 60. In thisexample, the user 24 wishes to change the first timestamp 66 of acomment from t=tj to t=tk (i.e., the user 24 wishes a comment to beassociated with the new point t=tk in the TV program 2 as opposed to thepoint t=tj that it is currently associated with).

At step s52, the user 24 selects the marker 68 (corresponding to thetimestamp to be edited) of the relevant comment box 58. This may beperformed in any appropriate way. For example, if the second UI 60 isdisplayed on a touch-screen display (e.g., the display 82 of the tabletcomputer 30), then the user 24 may select the marker 68 by touching it(e.g., with a finger). Also for example, the marker 68 may be selectedby the user 24 moving a mouse cursor over the marker and clicking amouse button.

At step s54, once the marker 68 has been selected, the user 24repositions the marker 68 (e.g., by using a “drag and drop” technique orby sliding the marker 68 along the top of its comment box 58) from aninitial position (where the marker 68 is aligned with the point t=tj onthe progress bar 54) to its new position (where the marker 68 is alignedwith the point t=tk on the progress bar 54).

FIG. 11 is a further schematic illustration (not to scale) of the secondUI 60. FIG. 11 shows the marker 68 of a comment box 58 moved from aninitial position (at t=tj and indicated in FIG. 11 by dotted lines) to anew position (at t=tk and indicated in FIG. 11 by solid lines). Themarker 68 is moved along such that it follows the solid arrow indicatedin FIG. 11 by the reference numeral 72. This may be performed in anyappropriate way. For example, if the second UI 60 is displayed on atouch-screen display, the user 24 may move the marker 68 by sliding hisfinger from the marker's original position (t=tj) to the marker's newposition (t=tk). Also for example, the marker 68 may be moved by theuser 24 by using a mouse controller and a “drag-and-drop” technique.

At step s56, the user 24 may de-select the marker 68 (e.g., byperforming a further action).

Thus, the first timestamp 66 of a comment may advantageously be alteredby a user 24 in a relatively simple and intuitive manner.

FIG. 12 is a process flow chart sowing certain steps of a method bywhich the user 24 may, using the second UI 60, associate a comment withmultiple first timestamps 66. In this example, the user 24 wishes tofurther associate a comment (that is initially associated with atime-step in the TV program 2 at t=ta) with a further additionaltime-step (at t=tb). In other words, the user 24 wishes a comment box 58to have two first timestamps (t=ta and t=tb), e.g., as shown in FIG. 13which is described below after the description of steps s58 through s62of FIG. 12.

At step s58, the user selects the marker 68 corresponding to the commentbox 58 he wishes to associate with the additional further time-step. Inother words, the user 24 selects the marker 68 corresponding to t=ta.This may be performed in any appropriate way, for example, as describedabove with reference to step s52 of FIG. 10.

At step s60, once the marker 68 has been selected, the user 24 creates aduplicate of the marker 68. This may be performed in any appropriateway. For example, the marker 68 may be copied using a conventional “copyand paste” technique or by performing some other action (e.g., tappingtwice on a touch-screen display).

At step s62, the user moves one copy of the marker 68 to a new desiredlocation. This may be performed using any appropriate process. Forexample, one or both copies of the marker 68 may be moved to desiredpositions by performing steps s52 through s56, as described above withreference to FIG. 10.

FIG. 13 is a further schematic illustration (not to scale) of the secondUI 60. The comment box 58 in FIG. 13 comprises two first timestamps 66and two corresponding markers 68 (at t=ta and t=tb), i.e., the commentbox 58 shown in FIG. 13 is a comment box resulting from the user 24performing the method of FIG. 12.

Thus, a single comment may be associated with a plurality of time-stepswithin the TV program 2 (e.g., by repeatedly performing the process ofFIG. 12). This may be done in an advantageously simple and intuitivemanner. This advantageously provides that a single comment box may beassociated with more than one point in a media presentation. This tendsto avoid the contents of comment boxes having to be duplicated.

The second UI 60 further provides an interface using which the user 24may view comments (made by himself or by other users) made in relationto part of the TV program 2. The part of the TV program may be specifiedby the user 24. An example method of performing such a process isdescribed below with reference to FIGS. 14 and 15.

FIG. 14 is a process flow chart showing certain steps of an examplemethod by which the user 24 may specify a part of the TV program 2 anddisplay, using the second UI 60, comments (and content items) thatrelate to that part (i.e., comments that have first timestamps 66 withinthe specified part of the TV program 2). In this example, the user 24wishes to display all comments that relate to a time in the TV programthat is between t=tc and t=td (i.e., the user 24 wishes to displaycomments that have a first timestamp 66 that is between t=tc and t=td).In this example, tc is a time-step that is later (in the TV program 2)than the start time 4 of the TV program 2. Also, td is a time-step thatis later (in the TV program 2) than tc. Also, td is a time-step that isearlier (in the TV program 2) than the end time 6 of the TV program 2.

At step s64, the user 24 selects a start (i.e., earliest) time-step forthe part of the TV program 2 that he wishes to view the comments made inrelation to. In this example, the user 24 selects the time-step t=tc.This may be performed in any appropriate way. For example, if the secondUI 60 is displayed on a touch-screen display (e.g., of the tabletcomputer 30), then the user 24 may select the start-time by touching theprogress bar 54 at the point corresponding to the desired start-time.The user 24 may change this selected time-step by, for example, slidinghis finger to a new position on the progress bar 54 corresponding to thedesired new start-time. Also for example, the start-time may be selectedby the user 24 moving a mouse cursor over the point on the progress bar54 corresponding to the desired start-time and clicking a mouse button.

At step s66, the user 24 selects an end (i.e., latest) time-step for thepart of the TV program 2 that he wishes to view the comments made inrelation to. In this example, the user 24 selects the time-step t=td.This may be performed in any appropriate way. For example, if the secondUI 60 is displayed on a touch-screen display (e.g., of the tabletcomputer 30), then the user 24 may select the end-time by touching theprogress bar 54 at the point corresponding to the desired end-time. Theuser 24 may change this selected time-step by, for example, sliding hisfinger to a new position on the progress bar 54 corresponding to thedesired new end-time. Also for example, the end-time may be selected bythe user 24 moving a mouse cursor over the point on the progress bar 54corresponding to the desired end-time, and clicking a mouse button.

Steps s64 and s66 may advantageously be performed simultaneously in arelatively simple and intuitive manner. For example, if the second UI 60is displayed on a touch-screen display (e.g., of the tablet computer30), then the user 24 may select the start-time by touching the progressbar 54 at that corresponding point with one digit (e.g., the user'sthumb), and simultaneously (or before or afterwards) the user 24 mayselect the end-time by touching the progress bar 54 at thatcorresponding point with another digit on the same hand (e.g., theuser's forefinger). The user 24 may advantageously change the selectedstart or end time by, for example, sliding either or both of his digitsalong the progress bar 54 so that they contact new positions on theprogress bar 54 corresponding to the desired new start and end times.The user 24 also change the length of the selected part by moving hisdigits (with which he has chosen the start and end times) closertogether or further part. Thus, the user 24 may use a relatively simple“pinching” action (on the progress bar 54) to specify, i.e., select(and, if desired by the user 24, alter) the start and end times of apart of the TV program 2.

Thus, the user 24 effectively specifies a range of time (i.e., tc to td)within the TV program 2.

At step s68, the tablet computer 30 displays all the comment boxes 58(and their respective contents) that have a first timestamp 66 that iswithin the specified range of time (i.e., between the start-time t=tcand the end-time t=td). A summary of the information contained in thosecomment boxes 58 may be displayed instead of or in addition to thecomment boxes 58 themselves. The comment boxes 58 that are displayed atstep s68 may be selected, e.g., from the set of all comment boxes 58, inany appropriate way, e.g., by the processor 80 of the tablet computer30.

FIG. 15 is a further schematic illustration (not to scale) of the secondUI 60. In FIG. 15, the shaded part 74 of the progress bar 54 relates tothe specified part (as specified at steps s64 and s66). Also in FIG. 15,the second UI 60 comprises a plurality of comment boxes 58, each ofwhich has a respective first timestamp 66 within the specified part 74.

Thus, a relatively simple process by which the user 24 can displaycomments (or a summary of the comments) relating to a specified part 74of the TV program 2 is provided. This advantageously provides a methodthat may be used to generate a summary of the part 74 of the TV program2. For example, the user 24 may just read the comments (or the summaryof the comments) relating to the specified portion 74 and gain anunderstanding of the events that occur in that portion 74, without theuser 24 having to watch (or consume) it.

Additional criteria that specify which comments or comment boxes 58(e.g., that relate to the portion 74) should be displayed to the user24, or how a summary of the comments that relate to the portion 74 isgenerated, may also be specified (e.g., by the user 24). For example,the user 24 may specify that he only wishes to have displayed to him, orsummarized to him, comments or comment boxes 58 that relate to theportion 74 and that have been made by members of a certain group ofother users. Also, for example, the user 24 may specify that he onlywishes to have displayed to him, or summarized to him, comments thathave been “liked” by other users.

Thus, a user may use a pinching or squeezing action on a timeline todefine a time interval. The system then summarizes or filters posts andcomments corresponding to that interval. For example, the system maydisplay a summary of all “liked” posts and comments that relate to apoint in that time interval. Also for example, a selection of videoclips associated with the posts and comments may be generated andcompiled into a video summary of the interval.

The selected interval may be stretched or shrunk (e.g., using a pinchingmotion), thereby changing the content that is displayed to the user 24.In this way, video editing may be performed, i.e., a video summary ofthe selected interval may be generated and changed by changing theinterval. This video summarization of the selected interval may be avideo summary of comments relating to the interval, i.e., the videosummarization may be driven by secondary content.

The information provided to the user 24 (i.e., the comments, summary ofthe comments, or summary of the video content) changes as the user 24changes the selected interval of the media presentation. Thisinformation may be presented to the user 24 in any appropriate way. Forexample, comments may ordered according to a rating given to thosecomments by other users. Also for example, a summary of posts andcomment may be presented to the user 24, e.g., as a text summary ofpostings and comments. Also for example, the harvesting or compiling ofvideo clips from the media presentation to create a video summary of theinterval may use user posts and comments to determine what visualmaterial to include in the video summary.

In the above embodiments, the above described methods are implemented inthe example network 14 described above with reference to FIG. 2.However, in other embodiments, the methods may be implemented in adifferent network that may be configured in a different way. Forexample, in a further example network 100, the modules of the examplenetwork 14 are configured to communicate with each other via theInternet 26. FIG. 16 is a schematic illustration (not to scale) of thefurther example network 100 in which the above described methods may beimplemented.

In view of the many possible embodiments to which the principles of thepresent invention may be applied, it should be recognized that theembodiments described herein with respect to the drawing figures aremeant to be illustrative only and should not be taken as limiting thescope of the invention. Therefore, the invention as described hereincontemplates all such embodiments as may come within the scope of thefollowing claims and equivalents thereof.

We claim:
 1. A method of displaying a plurality of presentationelements, the method comprising: displaying, on a user interface, aprogress bar, a length of the progress bar being representative of aduration of a multimedia presentation, a point along the length of theprogress bar corresponding to a point within the multimediapresentation; displaying, on the progress bar, a first point along thelength of the progress bar, the first point being distinct from an endof the progress bar; displaying, on the progress bar, a second pointalong the length of the progress bar, the second point being differentfrom the first point; and displaying, on the user interface,information, the information being dependent on one or morecommunications; wherein each communication has been composed in relationto a respective point or range within the multimedia presentation; andwherein the points or ranges within the multimedia presentation that thecommunications have been composed in relation to are between points inthe multimedia presentation that correspond to the first point and thesecond point.
 2. A method according to claim 1 wherein the displayedinformation is information selected from a group consisting of: one ormore of the communications, a summary of one or more of thecommunications, and a summary of the portion of the multimediapresentation that is between the first point and the second point, thesummary being dependent on one or more of the communications.
 3. Amethod according to claim 1: wherein each communication is displayed ina respective communication box; wherein each communication box comprisesa respective marker; and wherein the progress bar and the communicationboxes are displayed such that, for each communication box, the marker ofthat communication box is in alignment with a particular position alongthe length of the progress bar, the particular position corresponding tothe point or range in the multimedia presentation in relation to whichthe communication in that communication box was composed.
 4. A methodaccording to claim 1: wherein the first point has been selected, by auser, by selecting a first position along the length of the progressbar; and wherein the second point has been selected, by the user, byselecting a second position along the length of the progress bar.
 5. Amethod according to claim 4: wherein the user interface is displayed ona touch-screen display; wherein the selection of the first point by theuser has been made by the user touching the first position along thelength of the progress bar on the touch-screen display; and wherein theselection of the second point by the user has been made by the usertouching the second position along the length of the progress bar on thetouch-screen display.
 6. A method according to claim 1 furthercomprising: displaying, for each displayed communication, a date andtime at which that communication was composed.
 7. A method according toclaim 1: wherein the user interface is displayed on a display of anend-user device; and wherein the end-user device is arranged to: receivea selection, by a user of the end-user device, of the first point;receive a selection, by the user of the end-user device, of the secondpoint; and select from a group of one or more communications, thosecommunications that have been composed in relation to points or rangeswithin the multimedia presentation that are between points in themultimedia presentation that correspond to the first point and thesecond point.
 8. A method according to claim 1: wherein each of thecommunications comprise one or more content items; wherein each contentitem is selected from a group of content items consisting of mediaitems, multimedia items, references to media items, and references tomultimedia items; and wherein each content item has been selecteddepending on at least part of the communication and content informationrelating to content of the multimedia presentation.
 9. A user interfacecomprising: a progress bar, a length of the progress bar beingrepresentative of a duration of a multimedia presentation, a point alongthe length of the progress bar corresponding to a point within themultimedia presentation; a first point along the length of the progressbar, the first point being distinct from an end of the progress bar, thefirst point being different from a second point; the second point alongthe length of the progress bar; and information dependent on one or morecommunications; wherein each communication has been composed in relationto a respective point or range within the multimedia presentation; andwherein the points or ranges within the multimedia presentation that thecommunications have been composed in relation to are between points inthe multimedia presentation that correspond to the first point and thesecond point.
 10. A user interface according to claim 9 wherein thedisplayed information is information selected from a group consistingof: one or more of the communications, a summary of one or more of thecommunications, and a summary of the portion of the multimediapresentation that is between the first point and the second point, thesummary being dependent on one or more of the communications.
 11. A userinterface according to claim 9: wherein each communication is displayedin a respective communication box; wherein each communication boxcomprises a respective marker; and wherein the progress bar and thecommunication boxes are displayed such that, for each communication box,the marker of that communication box is in alignment with a particularposition along the length of the progress bar, the particular positioncorresponding to the point or range in the multimedia presentation inrelation to which the communication in that communication box wascomposed.
 12. A user interface according to claim 9: wherein the firstpoint has been selected by a user by selecting a first position alongthe length of the progress bar; and wherein the second point has beenselected by the user by selecting a second position along the length ofthe progress bar.
 13. A user interface according to claim 12: whereinthe user interface is displayed on a touch-screen display; wherein theselection of the first point by the user has been made by the usertouching the first position along the length of the progress bar on thetouch-screen display; and wherein the selection of the second point bythe user has been made by the user touching the second position alongthe length of the progress bar on the touch-screen display.
 14. A userinterface according to claim 9 wherein, for each displayedcommunication, a date and time at which that communication was composedis displayed.
 15. A user interface according to claim 9: wherein each ofthe communications comprise one or more content items; wherein eachcontent item is selected from a group of: content items consisting ofmedia items, multimedia items, references to media items, and referencesto multimedia items; and wherein each content item has been selecteddepending on at least part of the communication and content informationrelating to content of the multimedia presentation.
 16. A method ofprocessing one or more communications, each communication having beencomposed by an originator, each communication having been composed inrelation to a point or range within a multimedia presentation, themethod comprising: displaying, on a user interface, a progress bar, alength of the progress bar being representative of a duration of themultimedia presentation, a point along the length of the progress barcorresponding to a point within the multimedia presentation; receiving,by a first receiving module, a selection of a first point along thelength of the progress bar, the first point being distinct from an endof the progress bar; receiving, by a second receiving module, aselection of a second point along the length of the progress bar, thesecond point being different from the first point; selecting from agroup comprising one or more communications, by a selection moduleoperatively connected to the receiving modules, those communicationsthat have been composed in relation to points or ranges within themultimedia presentation that are between points in the multimediapresentation that correspond to the first point and the second point;processing, by a processor operatively connected to the selectionmodule, the selected communications to determine information dependentupon those selected communications; and displaying, on the userinterface, the determined information.
 17. Apparatus for processing oneor more communications, each communication having been composed by anoriginator, each communication having been composed in relation to apoint or range within a multimedia presentation, the apparatuscomprising: a display configured to display a progress bar, a length ofthe progress bar being representative of a duration of the multimediapresentation, a point along the length of the progress bar correspondingto a point within the multimedia presentation; a first receiving moduleconfigured to receive a selection of a first point along the length ofthe progress bar, the first point being distinct from an end of theprogress bar; a second receiving module configured to receive aselection of a second point along the length of the progress bar, thesecond point being different from the first point; a selection moduleoperatively connected to the receiving modules and configured to selectfrom a group comprising one or more communications, those communicationsthat have been composed in relation to points or ranges within themultimedia presentation that are between points in the multimediapresentation that correspond to the first point and the second point;and a processor operatively connected to the selection module andconfigured to process the selected communications to determineinformation dependent upon those selected communications; wherein thedisplay is further configured to display the determined information.